{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from cot import Collection\n",
    "from cot.stats import evaluation_as_table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /home/kon/nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n"
     ]
    }
   ],
   "source": [
    "full_prompt = Collection.from_json(\"./chatgpt_hard_full_prompt.json\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "full_prompt.select_generated_cots(cot_trigger=\"zhou-01\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "commonsense_qa {1}\n",
      "med_qa {1}\n",
      "open_book_qa {1}\n",
      "strategy_qa {1}\n",
      "worldtree {1}\n"
     ]
    }
   ],
   "source": [
    "full_prompt.number_generated_cots()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "full_prompt.full_text_prompts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "full_prompt.dump(\"./chatgpt_hard_full_prompt.json\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "ae =\"Rain is usually guaranteed when all are present but\\nA) cirrus clouds\\nB) cumulus clouds\\nC) hail stones\\nD) direct sunshine\\n\\nAnswer: Let's work this out in a step by step way to be sure we have the right answer.First, we need to understand what conditions are necessary for rain to occur. Rain is formed when water droplets in the atmosphere combine and become heavy enough to fall to the ground. This usually happens when warm, moist air rises and cools, causing the water vapor to condense into clouds. Eventually, the water droplets in the clouds become too heavy to stay aloft and fall to the ground as rain.\\n\\nNow, let's look at the options given:\\n\\nA) Cirrus clouds are thin, wispy clouds that are usually found at high altitudes. They are made up of ice crystals and are not associated with rain. Therefore, we can eliminate this option.\\n\\nB) Cumulus clouds are large, fluffy clouds that are often associated with fair weather. However, they can also develop into thunderstorms and produce rain. So, this option is a possibility.\\n\\nC) Hail stones are small, hard balls of ice that are formed in thunderstorms. They are not directly related to rain, but they can be a sign of severe weather. Therefore, we can eliminate this option.\\n\\nD) Direct sunshine is not a condition for rain to occur. In fact, sunny weather usually means that there is no rain. Therefore, we can eliminate this option.\\n\\nBased on our analysis, the correct answer is B) cumulus clouds. While not all cumulus clouds produce rain, they are often associated with rain and thunderstorms.\\nTherefore, among A through D, the answer is\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "ans = \"B) cumulus clouds.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Rain is usually guaranteed when all are present but\n",
      "A) cirrus clouds\n",
      "B) cumulus clouds\n",
      "C) hail stones\n",
      "D) direct sunshine\n",
      "\n",
      "Answer: Let's work this out in a step by step way to be sure we have the right answer.First, we need to understand what conditions are necessary for rain to occur. Rain is formed when water droplets in the atmosphere combine and become heavy enough to fall to the ground. This usually happens when warm, moist air rises and cools, causing the water vapor to condense into clouds. Eventually, the water droplets in the clouds become too heavy to stay aloft and fall to the ground as rain.\n",
      "\n",
      "Now, let's look at the options given:\n",
      "\n",
      "A) Cirrus clouds are thin, wispy clouds that are usually found at high altitudes. They are made up of ice crystals and are not associated with rain. Therefore, we can eliminate this option.\n",
      "\n",
      "B) Cumulus clouds are large, fluffy clouds that are often associated with fair weather. However, they can also develop into thunderstorms and produce rain. So, this option is a possibility.\n",
      "\n",
      "C) Hail stones are small, hard balls of ice that are formed in thunderstorms. They are not directly related to rain, but they can be a sign of severe weather. Therefore, we can eliminate this option.\n",
      "\n",
      "D) Direct sunshine is not a condition for rain to occur. In fact, sunny weather usually means that there is no rain. Therefore, we can eliminate this option.\n",
      "\n",
      "Based on our analysis, the correct answer is B) cumulus clouds. While not all cumulus clouds produce rain, they are often associated with rain and thunderstorms.\n",
      "Therefore, among A through D, the answer is B) cumulus clouds.\n"
     ]
    }
   ],
   "source": [
    "print(ae,ans)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "q = \"Which event occurs once per day?\\nA) The Moon rises.\\nB) The new Moon occurs.\\nC) The Moon cycles through its phases.\\nD) The Moon passes in front of the Sun.\\n\\nAnswer: Let's work this out in a step by step way to be sure we have the right answer.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "ae = \"A) The Moon rises: This event occurs once per day, but it is not specific to the Moon. The Sun also rises once per day.\\n\\nB) The new Moon occurs: This event occurs once per lunar month, which is about every 29.5 days.\\n\\nC) The Moon cycles through its phases: This event occurs once per lunar cycle, which is about every 29.5 days.\\n\\nD) The Moon passes in front of the Sun: This event occurs during a solar eclipse, which happens only a few times per year and is not specific to a daily occurrence.\\n\\nTherefore, the correct answer is C) The Moon cycles through its phases, which occurs once per day.\\nTherefore, among A through D, the answer is\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "ans = \"C) The Moon cycles through its phases.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "You are Hermes. Follow the instructions below and fill in the fields in the template.\n",
      "\n",
      "- user: Which event occurs once per day?\n",
      "A) The Moon rises.\n",
      "B) The new Moon occurs.\n",
      "C) The Moon cycles through its phases.\n",
      "D) The Moon passes in front of the Sun.\n",
      "\n",
      "Answer: Let's work this out in a step by step way to be sure we have the right answer.\n",
      "\n",
      "- Hermes: A) The Moon rises: This event occurs once per day, but it is not specific to the Moon. The Sun also rises once per day.\n",
      "\n",
      "B) The new Moon occurs: This event occurs once per lunar month, which is about every 29.5 days.\n",
      "\n",
      "C) The Moon cycles through its phases: This event occurs once per lunar cycle, which is about every 29.5 days.\n",
      "\n",
      "D) The Moon passes in front of the Sun: This event occurs during a solar eclipse, which happens only a few times per year and is not specific to a daily occurrence.\n",
      "\n",
      "Therefore, the correct answer is C) The Moon cycles through its phases, which occurs once per day.\n",
      "Therefore, among A through D, the answer isC) The Moon cycles through its phases.\n",
      "\n",
      "- user: Which event occurs once per day?\n",
      "A) The Moon rises.\n",
      "B) The new Moon occurs.\n",
      "C) The Moon cycles through its phases.\n",
      "D) The Moon passes in front of the Sun.\n",
      "\n",
      "Answer: Let's work this out in a step by step way to be sure we have the right answer.\n",
      " \n",
      "- Hermes initial response: {your response}\n",
      "- Hermes reflection on response: {evaluate your response for completeness and accuracy. What did you say wrong, what could you have said better}\n",
      "- Hermes Updated response: {Your updated improved response based on reflection}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "f\"\"\"\n",
    "You are Hermes. Follow the instructions below and fill in the fields in the template.\n",
    "\n",
    "- user: {q}\n",
    "\n",
    "- Hermes: {ae}{ans}\n",
    "\n",
    "- user: {q}\n",
    "\"\"\",\n",
    "\"\"\"\n",
    "- Hermes initial response: {your response}\n",
    "- Hermes reflection on response: {evaluate your response for completeness and accuracy. What did you say wrong, what could you have said better}\n",
    "- Hermes Updated response: {Your updated improved response based on reflection}\n",
    "\"\"\"\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
